Non Mean Square Error Criteria for the Training of Learning Machines
نویسنده
چکیده
In recent papers, Miller, Goodman & Smyth (1991, 1993) provided conditions on the cost function used for the training of a neural network in order to ensure that the output of the network approximates the conditional expectation of the desired output, given the input. However, they only considered the single-output case. In this paper, we provide another, rather straightforward, proof of the same results, for the general multi-outputs case; all the developments being presented in the context of estimation theory. More precisely, among a class of "reasonable" performance criteria, we provide necessary and sufficient conditions on the cost function so that the optimal estimate is the conditional expectation of the desired outputs, whatever the noise characteristics affecting the data. We furthermore provide a short overview of related results from estimation theory, and verify numerically the developments by comparing the optimal estimator of several performance criteria. We must stress that, while all these results are stated for a neural network, they are however true in general for any learning machine that is trained in order to predict an output y in function of an input x .
منابع مشابه
A Comparative Study of Extreme Learning Machines and Support Vector Machines in Prediction of Sediment Transport in Open Channels
The limiting velocity in open channels to prevent long-term sedimentation is predicted in this paper using a powerful soft computing technique known as Extreme Learning Machines (ELM). The ELM is a single Layer Feed-forward Neural Network (SLFNN) with a high level of training speed. The dimensionless parameter of limiting velocity which is known as the densimetric Froude number (Fr) is predicte...
متن کاملShort Term Load Forecasting Using Empirical Mode Decomposition, Wavelet Transform and Support Vector Regression
The Short-term forecasting of electric load plays an important role in designing and operation of power systems. Due to the nature of the short-term electric load time series (nonlinear, non-constant, and non-seasonal), accurate prediction of the load is very challenging. In this article, a method for short-term daily and hourly load forecasting is proposed. In this method, in the first step, t...
متن کاملSupport Vector Machines: Training and Applications Contents 1 Introduction 2 2 Support Vector Machines 2
The Support Vector Machine (SVM) is a new and very promising classiication technique developed by Vapnik and his group at AT&T Bell Laboratories 3, 6, 8, 24]. This new learning algorithm can be seen as an alternative training technique for Polynomial, Radial Basis Function and Multi-Layer Perceptron classiiers. The main idea behind the technique is to separate the classes with a surface that ma...
متن کاملGlobal Solar Radiation Prediction for Makurdi, Nigeria Using Feed Forward Backward Propagation Neural Network
The optimum design of solar energy systems strongly depends on the accuracy of solar radiation data. However, the availability of accurate solar radiation data is undermined by the high cost of measuring equipment or non-functional ones. This study developed a feed-forward backpropagation artificial neural network model for prediction of global solar radiation in Makurdi, Nigeria (7.7322 N lo...
متن کاملEvaluating machine learning methods and satellite images to estimate combined climatic indices
The reflections recorded on satellite images have been affected by various environmental factors. In these images, some of these factors are combined with other environmental factors that cannot be distinguished. Therefore, it seems wise to model these environmental phenomena in the form of hybrid indicators. In this regard, satellite imagery and machine learning methods can play a unique role ...
متن کامل